A subtle programming error in a commonly used application can sometimes become a serious security hole if exploited by malicious users. A traditional approach for detecting programming errors has been to visually examine the source code to detect loopholes. Manual inspection of the source code, however, is tedious and expensive in human costs and very time consuming when applied to systems that are very large. Moreover, traditional manual inspections can be erratic and depend largely on the level of expertise of the software engineers involved.
Additionally, most bugs in the software are contained in the error handling paths. Normal execution paths are usually well tested, but detection of bugs in the error handling path has been largely overlooked. Sometimes such bugs are difficult to detect or reproduce because they happen rarely, such as, only when resources of the system are depleted, like memory resources. Additionally, other errors in the program may have a delayed effect, for example, a buffer overflow may lead to heap corruption and cause a system crash much later. Thus, a need exists for a method to evaluate the robustness of the software and detect its potential errors and vulnerabilities in an effective manner.
Moreover, manual inspection requires access to the source code of the software. With the increasing pressure of cutting down the business cost, many companies utilize software products developed by third party vendors. Source code access is typically not available in this case.
One approach that has been used in the past is to use black box testing which passes various input values into the software and checks its output. Traditionally black box testing methods do not adequately test the behavior of the software on exceptional system conditions, such as when the system runs out of file descriptors.
A need exists for a way of exposing a large number of software bugs without source code access.
A need exists for an automatic process, which eliminates the manual methods for detecting errors in software.
A need exists for a method to detect programming errors in large software systems quickly.
The present embodiments meet these needs.